Fisheye lens graphical user interfaces

ABSTRACT

In an implementation, an appearance of a lens is applied to an original image by a data processing system to produce a presentation for display on a display screen, the appearance of the lens having a focal region at least partially surrounded by a shoulder region. A data interaction mode is provided by the data processing system to interact with underlying data of the focal region if a cursor is positioned over the focal region in the presentation. A lens interaction mode is provided by the data processing system to adjust one or more parameters of the appearance of the lens if the cursor is positioned over the shoulder region in the presentation. The presentation is displayed on the display screen.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority as a continuation from U.S. patentapplication Ser. No. 11/104,571 filed Apr. 13, 2005, which claimspriority to U.S. Provisional Patent Application No. 60/561,876, filedApr. 14, 2004, the entire disclosures of which are hereby incorporatedby reference

BACKGROUND

Modern computer graphics systems, including virtual environment systems,are used for numerous applications such as flight training,surveillance, and even playing computer games. In general, theseapplications are launched by the computer graphics system's operatingsystem upon selection by a user from a menu or other graphical userinterface (“GUI”). A GUI is used to convey information to and receivecommands from users and generally includes a variety of GUI objects orcontrols, including icons, toolbars, drop-down menus, text, dialogboxes, buttons, and the like. A user typically interacts with a GUI byusing a pointing device (e.g., a mouse) to position a pointer or cursorover an object and “clicking” on the object.

One problem with these computer graphics systems is their inability toeffectively display detailed information for selected graphic objectswhen those objects are in the context of a larger image. A user maydesire access to detailed information with respect to an object in orderto closely examine the object, to interact with the object, or tointerface with an external application or network through the object.For example, the detailed information may be a close-up view of theobject or a region of a digital map image.

While an application may provide a GUI for a user to access and viewdetailed information for a selected object in a larger image, in doingso, the relative location of the object in the larger image may be lostto the user. Thus, while the user may have gained access to the detailedinformation to interact with the object, the user may lose sight of thecontext within which that object is positioned in the larger image. Thisis especially so when the user interacts with the GUI using a computermouse or keyboard. The interaction may further distract the user fromthe context in which the detailed information is to be understood. Thisproblem is an example of what is often referred to as the “screen realestate problem”.

SUMMARY

According to one embodiment, there is provided a method for interactingwith a region-of-interest in an original image displayed on a displayscreen, comprising: applying a lens to the original image to produce apresentation for display on the display screen, the lens having a focalregion for the region-of-interest at least partially surrounded by ashoulder region; receiving one or more first signals to interact withthe region-of-interest when a cursor is positioned over the focal regionin the presentation; and, receiving one or more second signals to adjustthe lens through a graphical user interface (“GUI”) displayed over thelens when the cursor is positioned over the shoulder region in thepresentation.

According to another embodiment, there is provided a method forinteracting with a region-of-interest in an original image displayed ona display screen, comprising: applying a lens to the original image toproduce a presentation for display on the display screen, the lenshaving within a perimeter, a focal region for the region-of-interest atleast partially surrounded by a shoulder region; receiving one or morefirst signals to interact with the region-of-interest when a cursor ispositioned over one of the focal region and shoulder region in thepresentation; and, receiving one or more second signals to adjust thelens through a graphical user interface (“GUI”) displayed over the lenswhen the cursor is positioned over the perimeter of the lens in thepresentation.

According to another embodiment, there is provided a method forgenerating a presentation of a region-of-interest in an original imagefor display on a display screen, comprising: receiving one or more firstsignals to define a boundary for the region-of-interest in the originalimage; receiving one or more second signals to adjust the boundarythereby defining a lens for the region-of-interest, the lens havingwithin a perimeter, a focal region for the region-of-interest at leastpartially surrounded by a shoulder region, the perimeter having anextent and the focal region having a magnification both defined by theboundary; and, applying the lens to the original image to produce thepresentation.

According to another embodiment, there is provided a method forgenerating a presentation of a region-of-interest in an original imagefor display on a display screen, comprising: applying a lens to theoriginal image to produce a presentation for display on the displayscreen, the lens having a focal region for the region-of-interest atleast partially surrounded by a shoulder region, the focal and shoulderregions separated by a perimeter; and, receiving one or more signals toreposition the lens when a cursor is positioned in the focal region andpushed against the perimeter.

In accordance with further embodiments there is provided an apparatussuch as a data processing system, a method for adapting this system, aswell as articles of manufacture such as a computer readable mediumhaving program instructions recorded thereon for practicing the method.

BRIEF DESCRIPTION OF THE DRAWINGS

Further features and advantages of the embodiments will become apparentfrom the following detailed description, taken in combination with theappended drawings, in which:

FIG. 1 is a graphical representation of the geometry for constructing athree-dimensional perspective viewing frustum, relative to an x, y, zcoordinate system, in accordance with elastic presentation spacegraphics technology;

FIG. 2 is a graphical representation of the geometry of a presentationin accordance with elastic presentation space graphics technology;

FIG. 3 is a block diagram illustrating a data processing system;

FIG. 4 is a partial screen capture illustrating a GUI having lenscontrol elements for user interaction with detail-in-contextpresentations in accordance with an embodiment;

FIG. 5 is a partial screen capture illustrating a GUI having lenscontrol elements, including a scoop control slide bar icon, for userinteraction with detail-in-context presentations in accordance with anembodiment;

FIG. 6 is a partial screen capture illustrating a GUI having designatedregions for lens control and data interaction in accordance with anembodiment;

FIG. 7 is a partial screen capture illustrating the GUI of FIG. 6 inwhich shading is used to indicate the regions for lens control and datainteraction;

FIG. 8 is a partial screen capture illustrating a GUI having arrow iconspresented on the sides of a lens for positioning the lens in accordancewith an embodiment;

FIG. 9 is a screen capture illustrating a GUI in which a tab icon ispresented on a side of a lens for positioning the lens in accordancewith an embodiment;

FIG. 10 is a partial screen capture illustrating a GUI for specifyingand adjusting a lens through its focal region in accordance with anembodiment;

FIG. 11 is a partial screen capture illustrating a lens specified withthe GUI of FIG. 10 in accordance with an embodiment;

FIG. 12 is a partial screen capture illustrating a GUI having a buttonicon for switching between a current initial lens specification GUI anda subsequent lens adjustment GUI in accordance with an embodiment;

FIG. 13 is a partial screen capture illustrating the subsequent lensadjustment GUI presented upon selection of the button icon of the GUI ofFIG. 12 in accordance with an embodiment;

FIG. 14 is a partial screen capture illustrating a GUI for positioning alens in accordance with an embodiment;

FIG. 15 is a partial screen capture illustrating the repositioning of alens with the GUI of FIG. 14 in accordance with an embodiment; and

FIG. 16 is a flow chart illustrating operations of software moduleswithin the memory of the data processing system for interacting with aregion-of-interest in an original image displayed on a display screen,in accordance with an embodiment.

It will be noted that throughout the appended drawings, like featuresare identified by like reference numerals.

DETAILED DESCRIPTION

In the following description, numerous specific details are set forth.However, it is understood that the techniques described herein may bepracticed without these specific details. The term “data processingsystem” is used herein to refer to any machine for processing data,including the computer systems and network arrangements describedherein. The techniques may be implemented in any computer programminglanguage provided that the operating system of the data processingsystem provides the facilities that may support the techniques.

The “screen real estate problem” generally arises whenever large amountsof information are to be displayed on a display screen of limited size.Conventional tools that address this problem include panning andzooming. While these tools are suitable for a large number of visualdisplay applications, these conventional tools become less effectivewhere sections of the visual information are spatially related, such asin layered maps and three-dimensional representations, for example. Inthis type of information display, panning and zooming are not aseffective as much of the context of the panned or zoomed display may behidden.

One solution to this problem is the application of “detail-in-context”presentation techniques. Detail-in-context is the magnification of aparticular region-of-interest (the “focal region” or “detail”) in a datapresentation while preserving visibility of the surrounding information(the “context”). This technique has applicability to the display oflarge surface area media (e.g. digital maps) on computer screens ofvariable size including graphics workstations, laptop computers,personal digital assistants (“PDAs”), and cell phones.

In the detail-in-context discourse, differentiation is often madebetween the terms “representation” and “presentation”. A representationis a formal system, or mapping, for specifying raw information or datathat is stored in a computer or data processing system. For example, adigital map of a city is a representation of raw data including streetnames and the relative geographic location of streets and utilities.Such a representation may be displayed visually on a computer screen orprinted on paper. On the other hand, a presentation is a spatialorganization of a given representation that is appropriate for the taskat hand. Thus, a presentation of a representation organizes such thingsas the point of view and the relative emphasis of different parts orregions of the representation. For example, a digital map of a city maybe presented with a region magnified to reveal street names.

In general, a detail-in-context presentation may be considered as adistorted view (or distortion) of a portion of the originalrepresentation or image where the distortion is the result of theapplication of a “lens” like distortion function to the originalrepresentation. A detailed review of various detail-in-contextpresentation techniques such as “Elastic Presentation Space” (“EPS”) (or“Pliable Display Technology” (“PDT”)) may be found in a publication byMarianne S. T. Carpendale, entitled “A Framework for ElasticPresentation Space” (Carpendale, Marianne S. T., A Framework for ElasticPresentation Space (Burnaby, British Columbia: Simon Fraser University,1999)), which is incorporated herein by reference.

In general, detail-in-context data presentations are characterized bymagnification of areas of an image where detail is desired, incombination with compression of a restricted range of areas of theremaining information (i.e. the context), the result typically givingthe appearance of a lens having been applied to the display surface.Using the techniques described by Carpendale, points in a representationare displaced in three dimensions and a perspective projection is usedto display the points on a two-dimensional presentation display. Thus,when a lens is applied to a two-dimensional continuous surfacerepresentation, for example, the resulting presentation appears to bethree-dimensional. In other words, the lens transformation appears tohave stretched the continuous surface in a third dimension. In EPSgraphics technology, a two-dimensional visual representation is placedonto a surface; this surface is placed in three-dimensional space; thesurface, containing the representation, is viewed through perspectiveprojection; and the surface is manipulated to effect the reorganizationof image details. The presentation transformation is separated into twosteps: surface manipulation or distortion and perspective projection.

FIG. 1 is a graphical representation of the geometry 100 forconstructing a three-dimensional (“3D”) perspective viewing frustum 220,relative to an x, y, z coordinate system, in accordance with elasticpresentation space (EPS) graphics technology. In EPS technology,detail-in-context views of two-dimensional (“2D”) visual representationsare created with sight-line aligned distortions of a 2D informationpresentation surface within a 3D perspective viewing frustum 220. InEPS, magnification of regions of interest and the accompanyingcompression of the contextual region to accommodate this change in scaleare produced by the movement of regions of the surface towards theviewpoint (“VP”) 240 located at the apex of the pyramidal shape 220containing the frustum. The process of projecting these transformedlayouts via a perspective projection results in a new 2D layout whichincludes the zoomed and compressed regions. The use of the thirddimension and perspective distortion to provide magnification in EPSprovides a meaningful metaphor for the process of distorting theinformation presentation surface. The 3D manipulation of the informationpresentation surface in such a system is an intermediate step in theprocess of creating a new 2D layout of the information.

FIG. 2 is a graphical representation of the geometry 200 of apresentation in accordance with known EPS graphics technology. EPSgraphics technology employs viewer-aligned perspective projections toproduce detail-in-context presentations in a reference view plane 201which may be viewed on a display. Undistorted 2D data points are locatedin a basal plane 210 of a 3D perspective viewing volume or frustum 220which is defined by extreme rays 221 and 222 and the basal plane 210.The VP 240 is generally located above the centre point of the basalplane 210 and reference view plane (“RVP”) 201. Points in the basalplane 210 are displaced upward onto a distorted surface 230 which isdefined by a general 3D distortion function (i.e. a detail-in-contextdistortion basis function). The direction of the perspective projectioncorresponding to the distorted surface 230 is indicated by the lineFPo-FP 231 drawn from a point FPo 232 in the basal plane 210 through thepoint FP 233 which corresponds to the focus or focal region or focalpoint of the distorted surface 230. Typically, the perspectiveprojection has a direction 231 that is viewer-aligned (i.e., the pointsFPo 232, FP 233, and VP 240 are collinear).

EPS is applicable to multidimensional data and is suited toimplementation on a computer for dynamic detail-in-context display on anelectronic display surface such as a monitor. In the case of twodimensional data, EPS is typically characterized by magnification ofareas of an image where detail is desired 233, in combination withcompression of a restricted range of areas of the remaining information(i.e. the context) 234, the end result typically giving the appearanceof a lens 230 having been applied to the display surface. The areas ofthe lens 230 where compression occurs may be referred to as the“shoulder” 234 of the lens 230. The area of the representationtransformed by the lens may be referred to as the “lensed area”. Thelensed area thus includes the focal region and the shoulder. Toreiterate, the source image or representation to be viewed is located inthe basal plane 210. Magnification 233 and compression 234 are achievedthrough elevating elements of the source image relative to the basalplane 210, and then projecting the resultant distorted surface onto thereference view plane 201. EPS performs detail-in-context presentation ofn-dimensional data through the use of a procedure wherein the data ismapped into a region in an (n+1) dimensional space, manipulated throughperspective projections in the (n+1) dimensional space, and then finallytransformed back into n-dimensional space for presentation. EPS hasnumerous advantages over conventional zoom, pan, and scrolltechnologies, including the capability of preserving the visibility ofinformation outside 234 the local region-of-interest 233.

For example, and referring to FIGS. 1 and 2, in two dimensions, EPS canbe implemented through the projection of an image onto a reference plane201 in the following manner. The source image or representation islocated on a basal plane 210, and those regions of interest 233 of theimage for which magnification is desired are elevated so as to move themcloser to a reference plane situated between the reference viewpoint 240and the reference view plane 201. Magnification of the focal region 233closest to the RVP 201 varies inversely with distance from the RVP 201.As shown in FIGS. 1 and 2, compression of regions 234 outside the focalregion 233 is a function of both distance from the RVP 201, and thegradient of the function describing the vertical distance from the RVP201 with respect to horizontal distance from the focal region 233. Theresultant combination of magnification 233 and compression 234 of theimage as seen from the reference viewpoint 240 results in a lens-likeeffect similar to that of a magnifying glass applied to the image.Hence, the various functions used to vary the magnification andcompression of the source image via vertical displacement from the basalplane 210 are described as lenses, lens types, or lens functions. Lensfunctions that describe basic lens types with point and circular focalregions, as well as certain more complex lenses and advancedcapabilities such as folding, have previously been described byCarpendale.

FIG. 3 is a block diagram illustrating a data processing system 300adapted to implement an embodiment. The data processing system 300 issuitable for implementing EPS technology, for displayingdetail-in-context presentations of representations in conjunction with adetail-in-context graphical user interface (GUI) 400, as describedbelow, and for adjusting detail-in-context lenses in detail-in-contextpresentations. The data processing system 300 includes an input device310, a central processing unit (“CPU”) 320, memory 330, and a display340. The input device 310 may include a keyboard, a mouse, a pen andtablet, a trackball, an eye tracking device, a position tracking device,or a similar device. The CPU 320 may include dedicated coprocessors andmemory devices. The memory 330 may include RAM, ROM, databases, or diskdevices. And, the display 340 may include a computer screen, terminaldevice, or a hardcopy producing output device such as a printer orplotter. The data processing system 300 has stored therein datarepresenting sequences of instructions which when executed cause themethod described herein to be performed. Of course, the data processingsystem 300 may contain additional software and hardware.

Thus, the data processing system 300 includes computer executableprogrammed instructions for directing the system 300. The programmedinstructions may be embodied in one or more software modules 331resident in the memory 330 of the data processing system 300.Alternatively, the programmed instructions may be embodied on a tangiblecomputer readable medium (such as a CD disk or floppy disk) which may beused for transporting the programmed instructions to the memory 330 ofthe data processing system 300. Alternatively, the programmedinstructions may be embedded in a computer-readable, signal-bearingmedium that is uploaded to a network by a vendor or supplier of theprogrammed instructions, and this signal-bearing medium may bedownloaded through an interface to the data processing system 300 fromthe network by end users or potential buyers.

As mentioned, detail-in-context presentations of data using techniquessuch as pliable surfaces, as described by Carpendale, are useful inpresenting large amounts of information on limited-size displaysurfaces. Detail-in-context views allow magnification of a particularregion-of-interest (the “focal region”) 233 in a data presentation whilepreserving visibility of the surrounding information 210. In thefollowing, GUIs are described having lens control elements that can beimplemented in software and applied to the editing of digital images andto the adjustment lenses in detail-in-context presentations. Thesoftware can be loaded into and run by the data processing system 300 ofFIG. 3.

FIG. 4 is a partial screen capture illustrating a GUI 400 having lenscontrol elements for user interaction with detail-in-contextpresentations in accordance with an embodiment. Detail-in-contextpresentations are characterized by magnification of areas of an imagewhere detail is desired, in combination with compression of a restrictedrange of areas of the remaining information (i.e. the context), the endresult typically giving the appearance of a lens having been applied tothe display screen surface. This lens 410 includes a “focal region” 420having high magnification, a surrounding “shoulder region” 430 whereinformation is typically visibly compressed, and a “base” 412surrounding the shoulder region 430 and defining the extent of the lens410. In FIG. 4, the lens 410 is shown with a circular shaped base 412(or outline) and with a focal region 420 lying near the center of thelens 410. However, the lens 410 and focal region 420 may have anydesired shape. For example, in FIG. 5, the lens 410 has a pyramid shapewith a flat top 420 and trapezoidal shoulders 430. As mentioned above,the base of the lens 412 may be coextensive with the focal region 420.

In general, the GUI 400 has lens control elements that, in combination,provide for the interactive control of the lens 410. The effectivecontrol of the characteristics of the lens 410 by a user (i.e., dynamicinteraction with a detail-in-context lens) is advantageous. At a giventime, one or more of these lens control elements may be made visible tothe user on the display surface 340 by appearing as overlay icons on thelens 410. Interaction with each element is performed via the motion ofan input or pointing device 310 (e.g., a mouse) with the motionresulting in an appropriate change in the corresponding lenscharacteristic. As will be described, selection of which lens controlelement is actively controlled by the motion of the pointing device 310at any given time is determined by the proximity of the iconrepresenting the pointing device 310 (e.g. cursor) on the displaysurface 340 to the appropriate component of the lens 410. For example,“dragging” of the pointing device at the periphery of the boundingrectangle of the lens base 412 causes a corresponding change in the sizeof the lens 410 (i.e. “resizing”). Thus, the GUI 400 provides the userwith a visual representation of which lens control element is beingadjusted through the display of one or more corresponding icons.

For ease of understanding, the following description is in the contextof using a two-dimensional pointing device 310 that is a mouse, but itwill be understood that the techniques may be practiced with other 2D or3D (or even greater numbers of dimensions) pointing devices including atrackball, a pen and tablet, a keyboard, an eye tracking device, and aposition tracking device.

A mouse 310 controls the position of a cursor icon 401 that is displayedon the display screen 340. The cursor 401 is moved by moving the mouse310 over a flat surface, such as the top of a desk, in the desireddirection of movement of the cursor 401. Thus, the two-dimensionalmovement of the mouse 310 on the flat surface translates into acorresponding two-dimensional movement of the cursor 401 on the displayscreen 340.

A mouse 310 typically has one or more finger actuated control buttons(i.e. mouse buttons). While the mouse buttons can be used for differentfunctions such as selecting a menu option pointed at by the cursor 401,the disclosed techniques may use a single mouse button to “select” alens 410 and to trace the movement of the cursor 401 along a desiredpath. Specifically, to select a lens 410, the cursor 401 is firstlocated within the extent of the lens 410. In other words, the cursor401 is “pointed” at the lens 410. Next, the mouse button is depressedand released. That is, the mouse button is “clicked”. Selection is thusa point and click operation. To trace the movement of the cursor 401,the cursor 401 is located at the desired starting location, the mousebutton is depressed to signal the computer 320 to activate a lenscontrol element, and the mouse 310 is moved while maintaining the buttondepressed. After the desired path has been traced, the mouse button isreleased. This procedure is often referred to as “clicking” and“dragging” (i.e. a click and drag operation). It will be understood thata predetermined key on a keyboard 310 could also be used to activate amouse click or drag. In the following, the term “clicking” will refer tothe depression of a mouse button indicating a selection by the user andthe term “dragging” will refer to the subsequent motion of the mouse 310and cursor 401 without the release of the mouse button.

The GUI 400 may include the following lens control elements: move,pickup, resize base, resize focus, fold, magnify, zoom, and scoop. Eachof these lens control elements has at least one lens control icon oralternate cursor icon associated with it. In general, when a lens 410 isselected by a user through a point and click operation, the followinglens control icons may be displayed over the lens 410: pickup icon 450,base outline icon 412, base bounding rectangle icon 411, focal regionbounding rectangle icon 421, handle icons 481, 482, 491, 492 (see FIG.5) magnify slide bar icon 440, zoom icon 495, and scoop slide bar icon540 (see FIG. 5). Typically, these icons are displayed simultaneouslyafter selection of the lens 410. In addition, when the cursor 401 islocated within the extent of a selected lens 410, an alternate cursoricon 460, 470, 480, 490, 495 may be displayed over the lens 410 toreplace the cursor 401 or may be displayed in combination with thecursor 401. These lens control elements, corresponding icons, and theireffects on the characteristics of a lens 410 are described below withreference to FIGS. 4 and 5.

In general, when a lens 410 is selected by a point and click operation,bounding rectangle icons 411, 421 are displayed surrounding the base 412and focal region 420 of the selected lens 410 to indicate that the lens410 has been selected. With respect to the bounding rectangles 411, 421one might view them as glass windows enclosing the lens base 412 andfocal region 420, respectively. The bounding rectangles 411, 421 includehandle icons 481, 482, 491, 492 allowing for direct manipulation of theenclosed base 412 and focal region 420 as will be explained below. Thus,the bounding rectangles 411, 421 inform the user that the lens 410 hasbeen selected and also provide the user with indications as to whatmanipulation operations might be possible for the selected lens 410though use of the displayed handles 481, 482, 491, 492. Note that it iswell within the scope of the techniques described herein to provide abounding region having a shape other than generally rectangular. Such abounding region could be of any of a great number of shapes includingoblong, oval, ovoid, conical, cubic, cylindrical, polyhedral, spherical,etc.

Moreover, the cursor 401 provides a visual cue indicating the nature ofan available lens control element. As such, the cursor 401 willgenerally change in form by simply pointing to a different lens controlicon 450, 412, 411, 421, 481, 482, 491, 492, 440, 540. For example, whenresizing the base 412 of a lens 410 using a corner handle 491, thecursor 401 will change form to a resize icon 490 once it is pointed at(i.e. positioned over) the corner handle 491. The cursor 401 will remainin the form of the resize icon 490 until the cursor 401 has been movedaway from the corner handle 491.

Lateral movement of a lens 410 is provided by the move lens controlelement of the GUI 400. This functionality is accomplished by the userfirst selecting the lens 410 through a point and click operation. Then,the user points to a point within the lens 410 that is other than apoint lying on a lens control icon 450, 412, 411, 421, 481, 482, 491,492, 440, 540. When the cursor 401 is so located, a move icon 460 isdisplayed over the lens 410 to replace the cursor 401 or may bedisplayed in combination with the cursor 401. The move icon 460 informsthe user that the lens 410 may be moved and also provides the user withindications as to what movement operations are possible for the selectedlens 410. For example, the move icon 460 may include arrowheadsindicating up, down, left, and right motion. Next, the lens 410 is movedby a click and drag operation in which the user clicks and drags thelens 410 to the desired position on the screen 340 and then releases themouse button 310. The lens 410 is locked in its new position until afurther pickup and move operation is performed.

Lateral movement of a lens 410 is also provided by the pickup lenscontrol element of the GUI. This functionality is accomplished by theuser first selecting the lens 410 through a point and click operation.As mentioned above, when the lens 410 is selected a pickup icon 450 isdisplayed over the lens 410 near the centre of the lens 410. Typically,the pickup icon 450 will be a crosshairs. In addition, a base outline412 is displayed over the lens 410 representing the base 412 of the lens410. The crosshairs 450 and lens outline 412 inform the user that thelens has been selected and also provides the user with an indication asto the pickup operation that is possible for the selected lens 410.Next, the user points at the crosshairs 450 with the cursor 401. Then,the lens outline 412 is moved by a click and drag operation in which theuser clicks and drags the crosshairs 450 to the desired position on thescreen 340 and then releases the mouse button 310. The full lens 410 isthen moved to the new position and is locked there until a furtherpickup operation is performed. In contrast to the move operationdescribed above, with the pickup operation, it is the outline 412 of thelens 410 that the user repositions rather than the full lens 410.

Resizing of the base 412 (or outline) of a lens 410 is provided by theresize base lens control element of the GUI. After the lens 410 isselected, a bounding rectangle icon 411 is displayed surrounding thebase 412. For a rectangular shaped base 412, the bounding rectangle icon411 may be coextensive with the perimeter of the base 412. The boundingrectangle 411 includes handles 491. These handles 491 can be used tostretch the base 412 taller or shorter, wider or narrower, orproportionally larger or smaller. The corner handles 491 will keep theproportions the same while changing the size. The middle handles 492(see FIG. 5) will make the base 412 taller or shorter, wider ornarrower. Resizing the base 412 by the corner handles 491 will keep thebase 412 in proportion. Resizing the base 412 by the middle handles 492will change the proportions of the base 412. That is, the middle handles492 change the aspect ratio of the base 412 (i.e. the ratio between theheight and the width of the bounding rectangle 411 of the base 412).When a user points at a handle 491 with the cursor 401 a resize icon 490may be displayed over the handle 491 to replace the cursor 401 or may bedisplayed in combination with the cursor 401. The resize icon 490informs the user that the handle 491 may be selected and also providesthe user with indications as to the resizing operations that arepossible with the selected handle. For example, the resize icon 490 fora corner handle 491 may include arrows indicating proportional resizing.The resize icon (not shown) for a middle handle 492 may include arrowsindicating width resizing or height resizing. After pointing at thedesired handle 491, 492 the user would click and drag the handle 491,492 until the desired shape and size for the base 412 is reached. Oncethe desired shape and size are reached, the user would release the mousebutton 310. The base 412 of the lens 410 is then locked in its new sizeand shape until a further base resize operation is performed.

Resizing of the focal region 420 of a lens 410 is provided by the resizefocus lens control element of the GUI. After the lens 410 is selected, abounding rectangle icon 421 is displayed surrounding the focal region420. For a rectangular shaped focal region 420, the bounding rectangleicon 421 may be coextensive with the perimeter of the focal region 420.The bounding rectangle 421 includes handles 481, 482. These handles 481,482 can be used to stretch the focal region 420 taller or shorter, wideror narrower, or proportionally larger or smaller. The corner handles 481will keep the proportions the same while changing the size. The middlehandles 482 will make the focal region 420 taller or shorter, wider ornarrower. Resizing the focal region 420 by the corner handles 481 willkeep the focal region 420 in proportion. Resizing the focal region 420by the middle handles 482 will change the proportions of the focalregion 420. That is, the middle handles 482 change the aspect ratio ofthe focal region 420 (i.e. the ratio between the height and the width ofthe bounding rectangle 421 of the focal region 420). When a user pointsat a handle 481, 482 with the cursor 401 a resize icon 480 may bedisplayed over the handle 481, 482 to replace the cursor 401 or may bedisplayed in combination with the cursor 401. The resize icon 480informs the user that a handle 481, 482 may be selected and alsoprovides the user with indications as to the resizing operations thatare possible with the selected handle. For example, the resize icon 480for a corner handle 481 may include arrows indicating proportionalresizing. The resize icon 480 for a middle handle 482 may include arrowsindicating width resizing or height resizing. After pointing at thedesired handle 481, 482, the user would click and drag the handle 481,482 until the desired shape and size for the focal region 420 isreached. Once the desired shape and size are reached, the user wouldrelease the mouse button 310. The focal region 420 is then locked in itsnew size and shape until a further focus resize operation is performed.

Folding of the focal region 420 of a lens 410 is provided by the foldcontrol element of the GUI. In general, control of the degree anddirection of folding (i.e. skewing of the viewer aligned vector 231 asdescribed by Carpendale) is accomplished by a click and drag operationon a point 471, other than a handle 481, 482, on the bounding rectangle421 surrounding the focal region 420. The direction of folding isdetermined by the direction in which the point 471 is dragged. Thedegree of folding is determined by the magnitude of the translation ofthe cursor 401 during the drag. In general, the direction and degree offolding corresponds to the relative displacement of the focus 420 withrespect to the lens base 410. In other words, and referring to FIG. 2,the direction and degree of folding corresponds to the displacement ofthe point FP 233 relative to the point FPo 232, where the vector joiningthe points FPo 232 and FP 233 defines the viewer aligned vector 231. Inparticular, after the lens 410 is selected, a bounding rectangle icon421 is displayed surrounding the focal region 420. The boundingrectangle 421 includes handles 481, 482. When a user points at a point471, other than a handle 481, 482, on the bounding rectangle 421surrounding the focal region 420 with the cursor 401, a fold icon 470may be displayed over the point 471 to replace the cursor 401 or may bedisplayed in combination with the cursor 401. The fold icon 470 informsthe user that a point 471 on the bounding rectangle 421 may be selectedand also provides the user with indications as to what fold operationsare possible. For example, the fold icon 470 may include arrowheadsindicating up, down, left, and right motion. By choosing a point 471,other than a handle 481, 482, on the bounding rectangle 421 a user maycontrol the degree and direction of folding. To control the direction offolding, the user would click on the point 471 and drag in the desireddirection of folding. To control the degree of folding, the user woulddrag to a greater or lesser degree in the desired direction of folding.Once the desired direction and degree of folding is reached, the userwould release the mouse button 310. The lens 410 is then locked with theselected fold until a further fold operation is performed.

Magnification of the lens 410 is provided by the magnify lens controlelement of the GUI. After the lens 410 is selected, the magnify controlis presented to the user as a slide bar icon 440 near or adjacent to thelens 410 and typically to one side of the lens 410. Sliding the bar 441of the slide bar 440 results in a proportional change in themagnification of the lens 410. The slide bar 440 informs the user thatmagnification of the lens 410 may be selected and also provides the userwith an indication as to what level of magnification is possible. Theslide bar 440 includes a bar 441 that may be slid up and down, or leftand right, to adjust and indicate the level of magnification. To controlthe level of magnification, the user would click on the bar 441 of theslide bar 440 and drag in the direction of desired magnification level.Once the desired level of magnification is reached, the user wouldrelease the mouse button 310. The lens 410 is then locked with theselected magnification until a further magnification operation isperformed. In general, the focal region 420 is an area of the lens 410having constant magnification (i.e. if the focal region is a plane).Again referring to FIGS. 1 and 2, magnification of the focal region 420,233 varies inversely with the distance from the focal region 420, 233 tothe reference view plane (RVP) 201. Magnification of areas lying in theshoulder region 430 of the lens 410 also varies inversely with theirdistance from the RVP 201. Thus, magnification of areas lying in theshoulder region 430 will range from unity at the base 412 to the levelof magnification of the focal region 420.

Zoom functionality is provided by the zoom lens control element of theGUI. Referring to FIG. 2, the zoom lens control element, for example,allows a user to quickly navigate to a region-of-interest 233 within acontinuous view of a larger presentation 210 and then zoom in to thatregion-of-interest 233 for detailed viewing or editing. Referring toFIG. 4, the combined presentation area covered by the focal region 420and shoulder region 430 and surrounded by the base 412 may be referredto as the “extent of the lens”. Similarly, the presentation area coveredby the focal region 420 may be referred to as the “extent of the focalregion”. The extent of the lens may be indicated to a user by a basebounding rectangle 411 when the lens 410 is selected. The extent of thelens may also be indicated by an arbitrarily shaped figure that boundsor is coincident with the perimeter of the base 412. Similarly, theextent of the focal region may be indicated by a second boundingrectangle 421 or arbitrarily shaped figure. The zoom lens controlelement allows a user to: (a) “zoom in” to the extent of the focalregion such that the extent of the focal region fills the display screen340 (i.e. “zoom to focal region extent”); (b) “zoom in” to the extent ofthe lens such that the extent of the lens fills the display screen 340(i.e. “zoom to lens extent”); or, (c) “zoom in” to the area lyingoutside of the extent of the focal region such that the area without thefocal region is magnified to the same level as the extent of the focalregion (i.e. “zoom to scale”).

In particular, after the lens 410 is selected, a bounding rectangle icon411 is displayed surrounding the base 412 and a bounding rectangle icon421 is displayed surrounding the focal region 420. Zoom functionality isaccomplished by the user first selecting the zoom icon 495 through apoint and click operation When a user selects zoom functionality, a zoomcursor icon 496 may be displayed to replace the cursor 401 or may bedisplayed in combination with the cursor 401. The zoom cursor icon 496provides the user with indications as to what zoom operations arepossible. For example, the zoom cursor icon 496 may include a magnifyingglass. By choosing a point within the extent of the focal region, withinthe extent of the lens, or without the extent of the lens, the user maycontrol the zoom function. To zoom in to the extent of the focal regionsuch that the extent of the focal region fills the display screen 340(i.e. “zoom to focal region extent”), the user would point and clickwithin the extent of the focal region. To zoom in to the extent of thelens such that the extent of the lens fills the display screen 340 (i.e.“zoom to lens extent”), the user would point and click within the extentof the lens. Or, to zoom in to the presentation area without the extentof the focal region, such that the area without the extent of the focalregion is magnified to the same level as the extent of the focal region(i.e. “zoom to scale”), the user would point and click without theextent of the lens. After the point and click operation is complete, thepresentation is locked with the selected zoom until a further zoomoperation is performed.

Alternatively, rather than choosing a point within the extent of thefocal region, within the extent of the lens, or without the extent ofthe lens to select the zoom function, a zoom function menu with multipleitems (not shown) or multiple zoom function icons (not shown) may beused for zoom function selection. The zoom function menu may bepresented as a pull-down menu. The zoom function icons may be presentedin a toolbar or adjacent to the lens 410 when the lens is selected.Individual zoom function menu items or zoom function icons may beprovided for each of the “zoom to focal region extent”, “zoom to lensextent”, and “zoom to scale” functions described above. In thisalternative, after the lens 410 is selected, a bounding rectangle icon411 may be displayed surrounding the base 412 and a bounding rectangleicon 421 may be displayed surrounding the focal region 420. Zoomfunctionality is accomplished by the user selecting a zoom function fromthe zoom function menu or via the zoom function icons using a point andclick operation. In this way, a zoom function may be selected withoutconsidering the position of the cursor 401 within the lens 410.

The concavity or “scoop” of the shoulder region 430 of the lens 410 isprovided by the scoop lens control element of the GUI. After the lens410 is selected, the scoop control is presented to the user as a slidebar icon 540 (see FIG. 5) near or adjacent to the lens 410 and typicallybelow the lens 410. Sliding the bar 541 (see FIG. 5) of the slide bar540 results in a proportional change in the concavity or scoop of theshoulder region 430 of the lens 410. The slide bar 540 informs the userthat the shape of the shoulder region 430 of the lens 410 may beselected and also provides the user with an indication as to what degreeof shaping is possible. The slide bar 540 includes a bar 541 that may beslid left and right, or up and down, to adjust and indicate the degreeof scooping. To control the degree of scooping, the user would click onthe bar 541 of the slide bar 540 and drag in the direction of desiredscooping degree. Once the desired degree of scooping is reached, theuser would release the mouse button 310. The lens 410 is then lockedwith the selected scoop until a further scooping operation is performed.

A user may choose to hide one or more lens control icons 450, 412, 411,421, 481, 482, 491, 492, 440, 495, 540 shown in FIGS. 4 and 5 from viewso as not to impede the user's view of the image within the lens 410.This may be helpful, for example, during an editing or move operation. Auser may select this option through means such as a menu, toolbar, orlens property dialog box.

In addition, the GUI 400 may maintain a record of control elementoperations such that the user may restore pre-operation presentations.This record of operations may be accessed by or presented to the userthrough “Undo” and “Redo” icons 497, 498, through a pull-down operationhistory menu (not shown), or through a toolbar.

Thus, detail-in-context data viewing techniques allow a user to viewmultiple levels of detail or resolution on one display 340. Theappearance of the data display or presentation is that of one or morevirtual lenses showing detail 233 within the context of a larger areaview 210. Using multiple lenses in detail-in-context data presentationsmay be used to compare two regions of interest at the same time. Foldingenhances this comparison by allowing the user to pull the regions ofinterest closer together. Moreover, using detail-in-context technologysuch as PDT, an area of interest can be magnified to pixel levelresolution, or to any level of detail available from the sourceinformation, for in-depth review. The digital images may include graphicimages, maps, photographic images, or text documents, and the sourceinformation may be in raster, vector, or text form.

For example, in order to view a selected object or area in detail, auser can define a lens 410 over the object using the GUI 400. The lens410 may be introduced to the original image to form a presentationthrough the use of a pull-down menu selection, tool bar icon, etc. Usinglens control elements for the GUI 400, such as move, pickup, resizebase, resize focus, fold, magnify, zoom, and scoop, as described above,the user adjusts the lens 410 for detailed viewing of the object orarea. Using the magnify lens control element, for example, the user maymagnify the focal region 420 of the lens 410 to pixel quality resolutionrevealing detailed information pertaining to the selected object orarea. That is, a base image (i.e., the image outside the extent of thelens) is displayed at a low resolution while a lens image (i.e., theimage within the extent of the lens) is displayed at a resolution basedon a user selected magnification 440, 441.

In operation, the data processing system 300 employs EPS techniques withan input device 310 and GUI 400 for selecting objects or areas fordetailed display to a user on a display screen 340. Data representing anoriginal image or representation is received by the CPU 320 of the dataprocessing system 300. Using EPS techniques, the CPU 320 processes thedata in accordance with instructions received from the user via an inputdevice 310 and GUI 400 to produce a detail-in-context presentation. Thepresentation is presented to the user on a display screen 340. It willbe understood that the CPU 320 may apply a transformation to theshoulder region 430 surrounding the region-of-interest 420 to affectblending or folding in accordance with EPS technology. For example, thetransformation may map the region-of-interest 420 and/or shoulder region430 to a predefined lens surface, defined by a transformation ordistortion function and having a variety of shapes, using EPStechniques. Or, the lens 410 may be simply coextensive with theregion-of-interest 420.

The lens control elements of the GUI 400 are adjusted by the user via aninput device 310 to control the characteristics of the lens 410 in thedetail-in-context presentation. Using an input device 310 such as amouse, a user adjusts parameters of the lens 410 using icons and scrollbars of the GUI 400 that are displayed over the lens 410 on the displayscreen 340. The user may also adjust parameters of the image of the fullscene. Signals representing input device 310 movements and selectionsare transmitted to the CPU 320 of the data processing system 300 wherethey are translated into instructions for lens control.

Moreover, the lens 410 may be added to the presentation before or afterthe object or area is selected. That is, the user may first add a lens410 to a presentation or the user may move a pre-existing lens intoplace over the selected object or area. The lens 410 may be introducedto the original image to form the presentation through the use of apull-down menu selection, tool bar icon, etc.

By using a detail-in-context lens 410 to select an object or area fordetailed information gathering, a user can view a large area (i.e.,outside the extent of the lens 410) while focusing in on a smaller area(or within the focal region 420 of the lens 410) surrounding theselected object. This makes it possible for a user to accurately gatherdetailed information without losing visibility or context of the portionof the original image surrounding the selected object.

The above methods and GUIs for adjusting lenses may utilize modes toprovide interaction with underlying data or the lens, e.g., to determineif mouse clicks are to operate on a lens 410 or on the underlying data(i.e., the representation). For example, FIG. 5 is a partial screencapture illustrating a GUI 500 having lens control elements, including ascoop control slide bar icon 540, 541, for user interaction withdetail-in-context presentations in accordance with an embodiment. If auser clicks and drags on the lens focal region 420 while in “lens mode”,the lens 410 will be translated according to the mouse movements. If theuser clicks and drags on the lens focal region 420 while in anapplication specific mode such as “draw mode,” then editing of theunderlying representation will be enabled through the lens 410. Refer tothe above description of the move lens and pickup lens control elements.The switching of modes is typically performed by selecting from atoolbar or menu in which the modes are presented or by entering a modeselection command through a keyboard. Thus, modes (e.g., lens mode, drawmode, etc.) may be used to separate lens interactions from datainteractions, which may be beneficial in applications in which the useof such modes is desired to provide for explicit switching to performdifferent interactions. Thus, in one instance the GUIs 400, 500 of FIGS.4 and 5 may use of the focal region 420 and shoulder region 430 for lenscontrol. In an implementation, clicking the mouse 410 while the cursor401 is located in the focal region 420 causes the lens parameters to bechanged and cannot be simultaneously used to manipulate (e.g., edit) theunderlying data.

FIG. 6 is a partial screen capture illustrating a GUI 600 havingdesignated regions for lens control 430 and data interaction 420 inaccordance with an embodiment. In the GUI 600 of FIG. 6, the focalregion 420 of the lens 410 is reserved for direct interaction with theunderlying data using a tool currently selected by the user. The toolmay be a drawing tool or an editing tool, etc. The shoulder region 430is reserved for lens control. The focal region 420 and shoulder region430 are separated by the focal region bounding rectangle icon 421 andits handle icons 481, 482 for adjusting the focal region 420. In orderto move the lens 410, the user clicks and drags on a point in theshoulder region 430 of the lens 410. The currently selected tooloperates in the focal region 420 of the lens 410, whereas clickingelsewhere 430, 481, 482 in the lens 410 manipulates the lens 410 itself.Thus, the GUI 600 provides switching between lens interaction and datainteraction modes. With the GUI 600, a user can seamlessly switch from amode for adjusting lens parameters to a mode for interacting withrepresentation data. In general, GUI lens control elements are notlocated in the focal region 420 (or alternatively in the lensed area420, 430) which frees up this region to be used for data manipulation.Thus, lens interaction is possible regardless of what application tool(e.g., draw, edit, cut, paste, etc.) the user has selected.

FIG. 7 is a partial screen capture illustrating the GUI 600 of FIG. 6 inwhich shading is used to indicate the regions for lens control 430, 421,481, 482 and data interaction 420. In FIG. 7, the focal region 420 isshaded in a first tone to identify it as a region for data interaction,the focal region bounding rectangle and handle icons 421, 481, 492, areshaded in a second tone to identify them as regions for lens control(i.e., focal region control), and the shoulder region 430 is shaded in athird tone to identify it as a region for lens control (i.e., lensmovement).

FIG. 8 is a partial screen capture illustrating a GUI 800 having arrowicons 801, 802, 803, 804 presented on the sides of a lens 410 forpositioning the lens 410 in accordance with an embodiment. According tothe embodiment of FIG. 8, the entire region inside the bounds of thelens (the lensed region 420, 430) is reserved for interaction with thedata, as such, new lens control elements are provided for adjusting theposition of the lens 410 and for adjusting parameters for the focalregion 420. These control elements are displayed outside the bounds 412of the lens 410 so that the interior of the lens 420, 430 can be usedfor data interaction. In FIG. 8, each side of the bounding rectangleicon 411 for the base 412 of the lens 410 includes an arrow icon 801,802, 803, 804 for positioning the lens 410. Each arrow icon 801, 802,803, 804 is used to move the lens 410 in the one-dimensional directionin which the arrow is pointing (i.e., up 801, left 802, down 803, right804). Alternatively, each arrow icon 801, 802, 803, 804 can be used tomove the lens 410 in two-dimensions. The lens 410 is moved by a clickand drag operation in which the user clicks and drags one of the arrowicons 801, 802, 803, 804 to a desired position on the screen 340 andthen releases the mouse button 310. A slide bar 440 is used to adjustthe level of magnification for the lens 410. Handles 491 on the boundingrectangle icon 411 for the base 412 of the lens 410 are used to resizethe bounds 412 of the lens 410.

Independent techniques are not provided to resize the focal region 420in this instance, rather, the handles 491, 492 on the bounding rectangleicon 411 for the base 412 also function to resize the focal region 420.In other words, resizing of the bounds 411 of the focal region 420 iscoupled to resizing of the bounds 412 of the base 412 of the lens 410.As the size (i.e., area) of lens base 412 increases, the size of thefocal region 420 increases correspondingly. Likewise, as the size of thelens base 412 decreases, the size of the focal region 420 decreasescorrespondingly. According to one embodiment, a user may chose therelationship between resizing of the base and focal region. For example,a menu item may be provided for selecting a constant ratio between thesizes of the base and the focal region (e.g., given a predeterminedlimit for the size of the base). A second menu item may be provided forselecting a constant absolute difference in size between the size of thebase and the focal region (e.g., given a predetermined limit for thesize of the base).

FIG. 9 is a screen capture illustrating a GUI 900 in which a tab icon905 is presented on a side of a lens 410 for positioning the lens 410 inaccordance with an embodiment. According to the embodiment of FIG. 9,the entire region inside the bounds of the lens (the lensed region 420,430) is reserved for interaction with the data, as such, new lenscontrol elements are provided for adjusting the position of the lens 410and for adjusting parameters for the focal region 420. These controlelements are displayed outside the bounds 412 of the lens 410 so thatthe interior of the lens 420, 430 can be used for data interaction. InFIG. 9, one side of the bounding rectangle icon 411 for the base 412 ofthe lens 410 includes a tab icon 905 for positioning the lens 410. Thetab icon 905 is used to move the lens 410 in two-dimensions. The lens410 is moved by a click and drag operation in which the user clicks anddrags on the tab icon 905 to a desired position on the screen 340 andthen releases the mouse button 310. A slide bar 440 is used to adjustthe level of magnification for the lens 410. Handles 491, 492 on thebounding rectangle icon 411 for the base 412 of the lens 410 are used toresize the bounds 412 of the lens 410. Independent means are notprovided to resize the focal region 420 in this instance, rather, thehandles 491 on the bounding rectangle icon 411 for the base 412 alsofunction to resize the focal region 420. In other words, resizing of thebounds 411 of the focal region 420 is coupled to resizing of the bounds412 of the base 412 of the lens 410. As the size (i.e., area) of lensbase 412 increases, the size of the focal region 420 increasescorrespondingly. Likewise, as the size of the lens base 412 decreases,the size of the focal region 420 decreases correspondingly. As with theembodiment of FIG. 8, a user may chose the relationship between resizingof the base and focal region by selecting from a menu.

FIG. 10 is a partial screen capture illustrating a GUI 1000 forspecifying and adjusting a lens through its focal region in accordancewith an embodiment. FIG. 11 is a partial screen capture illustrating alens specified with the GUI 1000 of FIG. 10 in accordance with anembodiment. The GUI 1000 of FIG. 10 includes a user defined boundingrectangle icon 1001 for a region-of-interest 1005 in the representationthat will become the focal region 420 of the lens 410 presentation shownin FIG. 11. The GUI 1000 is used to specify a lens 410 as follows.First, an undistorted representation 1010 is displayed to a user on thedisplay screen 340. Second, the user identifies a region-of-interest1005 in the representation 1010 that the user would like to havemagnified. Third, the user activates a software module 331 (e.g., byselection a menu item in a pull down menu, etc.) allowing the user todraw a bounding rectangle icon 1001 around the region-of-interest 1005by moving a cursor 401 on the display screen 340 with a pointing device310 such as a mouse. Of course, the bounding rectangle icon 1001 couldhave any other shape (e.g., a circle, polygon, etc.). Fourth, the useradjusts the bounding rectangle icon 1001 to specify a magnification forthe region-of-interest 1005. The magnification so specified is used togenerate a detail-in-context lens presentation 1110 (see FIG. 11) havinga lens 410 with a focal region 420 sized in accordance with thespecified magnification. In other words, by “pulling” or “pushing” on aside of the bounding rectangle icon 1001 with a cursor 401 and mouse310, the user specifies the size (i.e., area) of the focal region 420which defines a magnification for the region-of-interest 1005. As thefocal region 420 expands (or contracts), the lens bounds 412 expands (orcontracts) correspondingly.

Advantageously, the GUI 1000 of FIGS. 10 and 11 provides a simpletechnique for a user to select a region-of-interest 1005 and thenquickly magnify that region-of-interest to a desired level. The size ofthe focal region 420, and hence the magnification of the lens, isadjusted directly by input from the user. That is, by pulling or pushingon a side of the bounding rectangle icon 1001 with a cursor 401 andmouse 310, the size of the focal region 420, magnification of the lens,and extent 412 of the lens 410 are specified. Once the lens 410 iscreated, as shown in the presentation of FIG. 11, the GUI 1000 can beused to further adjust the focal region 420 and hence the lens 410.

Note that with the GUI 1000 of FIGS. 10 and 11, changing the size of thefocal region 420 changes the size of objects presented in the focalregion 420. That is, focal region size is related to magnification. Incontrast, with the GUIs 400, 500 of FIGS. 4 and 5, changing the size ofthe focal region 420 changes the amount of information from the originalrepresentation presented in the focal region 420. That is, focal regionsize is not related to magnification (i.e., magnification remainsconstant when the size of the focal region is adjusted).

FIG. 12 is a partial screen capture illustrating a GUI 1200 having abutton icon 1201 for switching between a current initial lensspecification GUI 1000 and a subsequent lens adjustment GUI 500 inaccordance with an embodiment. FIG. 13 is a partial screen captureillustrating the subsequent lens adjustment GUI 500 presented uponselection of the button icon 1201 of the GUI 1200 of FIG. 12 inaccordance with an embodiment. The method of specifying and adjusting alens 410 using the GUI 1000 of FIGS. 10 and 11 can be an initial step ina user interaction with the lens 410. By this, it is meant that thisinitial step occurs once. After this initial step, the lens controlelements associated with the GUIs 400, 500 of FIGS. 4 and 5 are used toadjust the lens 410.

The GUI 1200 of FIG. 12 includes on the bounding rectangle icon 1001 ofFIGS. 10 and 11 a button icon 1201 for selection by a user to indicateto software modules 331 in the data processing system 300 that initialspecification of the lens 410 is complete. In FIG. 12, the button icon1201 is a “close box” icon 1201. Rather than including the button icon1201 on the bounding rectangle icon 1001, a menu item in a pull-downmenu or an icon in a toolbar could be provided as a means for indicatingto the system 300 that the initial specification of the lens 410 hasbeen completed. Upon selecting the button icon 1201, the GUI 1200 ofFIG. 12 transitions to the GUI 500 of FIG. 5 as shown in FIG. 13.

According to one embodiment, the GUI 1000 of FIGS. 10 and 11 functionsas both the initial technique to specify the lens 410 and the subsequenttechnique for adjusting the lens 410. In this embodiment, the GUI 1000can include one or more of the lens control elements described withrespect to the GUIs 400, 500, 600, 700, 800, 900 of FIGS. 4-9.

FIG. 14 is a partial screen capture illustrating a GUI 1400 forpositioning a lens 410 in accordance with an embodiment. In FIG. 14, thelens 410 has a circular shaped focal region 420, shoulder region 430,and base 412. However, the lens 410 may be of any other shape (e.g.,rectangular, etc.). According to this embodiment, the position of thelens 410 depends on the position of the cursor 401 within thepresentation 1401.

If the cursor 401 is repositioned (i.e., by moving a mouse 310) withinthe perimeter 1405 of the focal region 420 of the lens 410, the lens 410remains in place. Movement of the cursor 401 within the perimeter 1405of the focal region 420 is indicated by the single arrow 1410 in FIG.14. However, if the cursor 401 is placed in contact with the inner sideof the perimeter 1405 of the focal region 420, subsequent movement ofthe cursor 401 causes the lens 410 to be correspondingly repositioned.

FIG. 15 is a partial screen capture illustrating the repositioning of alens with the GUI 1400 of FIG. 14 in accordance with an embodiment. InFIG. 15, the cursor 401 is shown in contact with the inner side of theperimeter 1405 of the focal region 420. As so positioned, the directionof movement of the lens 410 upon subsequent movement of the cursor 410is indicated by the multiple arrows 1510. Thus, the lens 410 may berepositioned within the presentation 1401 when the cursor 401 is“pushed” against the inner side of the perimeter 1405 of the focalregion 420.

With the GUI 1400 of FIGS. 14 and 15, magnification for the lens 410 canbe adjusted using a scroll wheel provided on the mouse 310, usingcommands entered through a keyboard, or by selection from a toolbar ormenu. If the magnification level is reduced to zero, the lens 410 willnot be included in the presentation 1401 and hence manipulation of thecursor 401 within the presentation 1401 will not have an effect on lenspositioning.

When the pointing device 310 is a mouse or trackball, the position ofthe cursor 410 remains under the control of the pointing device. Thatis, the cursor 401 does not travel instantly from point to point, as itcould were the pointing device a pen or stylus input device. Thus, witha mouse input device, the position of the lens 410 in the presentation1401 is governed by the movement of the cursor 410 by the mouse withinthe focal region 420. As shown in FIGS. 14 and 15, moving the cursor 401about within the perimeter 1405 of the focal region 420 does not movethe lens, whereas moving the cursor 401 against the inner side of theperimeter 1405 of the focal region 420 causes the lens 410 to berepositioned such that the cursor 401 does not travel outside of thefocal region 420.

If the pointing device 310 is a pen or stylus input device (such asthose used with computer tablets), the device 310 may be lifted from thetablet, moved to a different position, and then returned to the surfaceof the tablet. Correspondingly, the cursor 401 disappears from thedisplay 340 when the pen 310 is lifted and reappears when the pen isreturned to the surface of the tablet. According to one embodiment, whenthe pen 310 is lifted and moved to a new position, the lens 410 isrepositioned correspondingly at the new position of the cursor 401.According to another embodiment, and referring to FIGS. 14 and 15, ifthe pen 310 is lifted and repositioned such that the cursor 401 ispositioned outside of the perimeter 1405 of the focal region 420 of thelens 410, the lens 410 remains stationary until the cursor 401 isreturned to within the perimeter 1405 of the focal region 420. In bothof these pen related embodiments, the lens 410 is repositioned when thecursor 401 is pushed against the inner side of the perimeter 1405 of thefocal region 420. Magnification for the lens 410 can be adjusted usingcommands entered through a keyboard, or by selection from a toolbar ormenu.

The above described method (i.e., relating to FIGS. 6 and 7) may besummarized with the aid of a flowchart. FIG. 16 is a flow chartillustrating operations 1600 of software modules 331 within the memory330 of the data processing system 300 for interacting with aregion-of-interest in an original image displayed on a display screen340, in accordance with an embodiment.

At step 1601, the operations 1600 start.

At step 1602, a lens 410 is applied to the original image to produce apresentation for display on the display screen 340, the lens 410 havinga focal region 420 for the region-of-interest at least partiallysurrounded by a shoulder region 430.

At step 1603, one or more first signals are received to interact withthe region-of-interest when a cursor 401 is positioned over the focalregion 420 in the presentation.

At step 1604, one or more second signals are received to adjust the lens410 through a graphical user interface (“GUI”) 600 displayed over thelens 410 when the cursor 401 is positioned over the shoulder region 430in the presentation.

At step 1605, the operations 1600 end.

The step of applying 1602 may further include displacing the originalimage onto the lens 230, 410 and perspectively projecting the displacingonto a plane 201 in a direction 231 aligned with a viewpoint 240 for theregion-of-interest 233. The method may further include displaying thepresentation on the display screen 340. The lens 410 may be a surface230. The GUI 600 may include functionality to adjust at least one of: amagnification for the focal region; a degree of scooping for theshoulder region; a size and a shape for the focal region; a size and ashape for a perimeter of the lens; or a location for the lens within theoriginal image. At least some of this functionality may be implementedusing icons 440, 540, 481, 482, 491, 492. The method may further includereceiving the one or more first and second signals and signals toposition the cursor 401 on the display screen 340 from a pointing device310 manipulated by a user. The pointing device 310 may be configured asat least one of a mouse, a pen and tablet, a trackball, a keyboard, aneye tracking device, and a position tracking device. The method mayfurther include applying first and second shades to the focal region 420and shoulder region 430, respectively, the first shade indicating thatthe focal region is for editing the original image and the second shadeindicating that the shoulder region is for adjusting the lens 410.

While these techniques were primarily discussed as a method, it shouldbe readily understood that the apparatus discussed above with referenceto a data processing system 300, may be programmed to enable thepractice of the method. Moreover, an article of manufacture for use witha data processing system 300, such as a pre-recorded storage device orother similar tangible computer readable medium including programinstructions recorded thereon, may direct the data processing system 300to facilitate the practice of the method. It is understood that suchapparatus and articles of manufacture also come within the scope.

In particular, the sequences of instructions which when executed causethe method described herein to be performed by the data processingsystem 300 of FIG. 3 can be contained in a tangible data carrier productaccording to one embodiment. This data carrier product can be loadedinto and run by the data processing system 300 of FIG. 3. In addition,the sequences of instructions which when executed cause the methoddescribed herein to be performed by the data processing system 300 ofFIG. 3 can be contained in a computer software product according to oneembodiment. This computer software product can be loaded into and run bythe data processing system 300 of FIG. 3. Moreover, the sequences ofinstructions which when executed cause the method described herein to beperformed by the data processing system 300 of FIG. 3 can be containedin an integrated circuit product including a coprocessor or memoryaccording to one embodiment ntion. This integrated circuit product canbe installed in the data processing system 300 of FIG. 3.

The embodiments described above are intended to be exemplary. The scopeof the techniques described herein is therefore intended to be limitedsolely by the scope of the following claims.

1. A method comprising: applying an appearance of a lens to an image bya data processing system to produce a presentation for display on adisplay screen, the appearance of the lens having a focal region atleast partially surrounded by a shoulder region; providing a datainteraction mode by the data processing system to interact withunderlying data of the focal region responsive to positioning a cursorover the focal region in the presentation; providing a lens interactionmode by the data processing system to adjust one or more parameters ofthe appearance of the lens responsive to positioning of the cursor overthe shoulder region in the presentation; and displaying the presentationon the display screen.
 2. The method of claim 1 wherein the focal regionincludes a magnification and the shoulder region includes amagnification that provides a transition from the magnification of thefocal region to the image.
 3. The method of claim 1 wherein the applyingfurther comprises displacing the image onto the appearance of the lensand perspectively projecting the displacing onto a plane in a directionaligned with a viewpoint for the focal region.
 4. The method of claim 3wherein the appearance of the lens is a surface.
 5. The method of claim1 wherein the lens interaction mode causes a graphical user interface(GUI) to be output to adjust the one or more parameters.
 6. The methodof claim 1 wherein the cursor is positioned on the display screen inresponse to signals received from a pointing device.
 7. The method ofclaim 6 wherein the pointing device is at least one of a mouse, a penand tablet, a trackball, a keyboard, an eye tracking device, or aposition tracking device.
 8. A method comprising: applying an appearanceof a lens to an image by a data processing system to produce apresentation for display on a display screen, the appearance of the lenshaving a perimeter; providing a data interaction mode by the dataprocessing system to interact with underlying data within the appearanceof the lens responsive to a cursor being positioned within theperimeter; providing a lens interaction mode by the data processingsystem to adjust one or more parameters of the appearance of the lensresponsive to the cursor being positioned over the perimeter of theappearance of the lens in the presentation; and displaying thepresentation on the display screen.
 9. The method of claim 8 wherein theappearance of the lens includes a focal region having a magnificationand a shoulder region having a magnification that provides a transitionfrom the magnification of the focal region to the image.
 10. The methodof claim 8 wherein the applying further comprises displacing the imageonto the appearance of the lens and perspectively projecting thedisplacing onto a plane in a direction aligned with a viewpoint for thefocal region.
 11. The method of claim 10 wherein the appearance of thelens is a surface.
 12. The method of claim 8 wherein the lensinteraction mode causes a graphical user interface (GUI) to be output toadjust the one or more parameters.
 13. The method of claim 8 wherein thecursor is positioned on the display screen in response to signalsreceived from a pointing device.
 14. The method of claim 13 wherein thepointing device is at least one of a mouse, a pen and tablet, atrackball, a keyboard, an eye tracking device, or a position trackingdevice.
 15. An article of manufacture including a computer-readablemedium having instruction stored thereon that, responsive to executionby a data processing system, cause the data processing system to performoperations comprising: applying an appearance of a lens to an image toproduce a presentation for display on a display screen, the appearanceof the lens having a focal region at least partially surrounded by ashoulder region; and responsive to repositioning of a cursor from thefocal region in the presentation to the shoulder region, switching froma data interaction mode that is configured to provide interaction withunderlying data of the focal region to a lens interaction mode that isconfigured to provide adjustment of one or more parameters of theappearance of the lens.
 16. The article of manufacture of claim 15wherein the focal region has a magnification and the shoulder region hasa magnification that provides a transition from the magnification of thefocal region to the image.
 17. The article of manufacture of claim 15wherein the applying further comprises displacing the image onto theappearance of the lens and perspectively projecting the displacing ontoa plane in a direction aligned with a viewpoint for the focal region.18. The article of manufacture of claim 15 wherein the lens interactionmode causes a graphical user interface (GUI) to be output to adjust theone or more parameters.
 19. The article of manufacture of claim 15wherein the cursor is positioned in response to signals received from apointing device.
 20. The article of manufacture of claim 19 wherein thepointing device is at least one of a mouse, a pen and tablet, atrackball, a keyboard, an eye tracking device, or a position trackingdevice.
 21. An article of manufacture including a computer-readablemedium having instruction stored thereon that, responsive to executionby a data processing system, cause the data processing system to performoperations comprising: applying an appearance of a lens to an image toproduce a presentation for display on a display screen, the appearanceof the lens having a perimeter; responsive to repositioning of a cursorfrom within the appearance of the lens to the perimeter, switching froma data interaction mode that is configured to provide interaction withunderlying data of the focal region to a lens interaction mode that isconfigured to provide adjustment of one or more parameters of theappearance of the lens.
 22. The article of manufacture of claim 21wherein the appearance of the lens includes a focal region having amagnification and a shoulder region having a magnification that providesa transition from the magnification of the focal region to the image.23. The article of manufacture of claim 22 wherein the applying furthercomprises displacing the image onto the appearance of the lens andperspectively projecting the displacing onto a plane in a directionaligned with a viewpoint for the focal region.
 24. The article ofmanufacture of claim 21 wherein the appearance of the lens is a surface.25. The article of manufacture of claim 21 wherein the lens interactionmode causes a graphical user interface (GUI) to be output to adjust theone or more parameters.
 26. The article of manufacture of claim 21wherein the cursor is positioned in response to signals received from apointing device.
 27. The article of manufacture of claim 26 wherein thepointing device is at least one of a mouse, a pen and tablet, atrackball, a keyboard, an eye tracking device, or a position trackingdevice.